Accounting for Imputation When Estimating Variances in the Economic Surveys at the Census Bureau

In most surveys, we encounter missing data of one or more types. Some sample units leave some data items blank--item nonresponse. The usual approach is to impute all missing, inconsistent, or otherwise invalid items. This paper considers the effect on the estimates of variance from treating imputed values as if they were reported, and compares strategies to address it. Data processing for many of the economic surveys conducted by the Census Bureau has been moved onto a generalized system called the Standard Economic Processing System (StEPS). Methods for estimating variances available in many systems, including StEPS, treat all processed data as if they were reported, ignoring the fact that the imputed values were not observed. The result is that this “naïve” estimator of variance typically is biased; often it underestimates the true variance. Under StEPS, a survey is not restricted to one method of imputation. Some surveys apply a primary method, and, if the variable(s) required is not available, they revert to a second or even a third method. We call this multi-phased procedure mixed imputation. We have not seen it addressed often in the literature beyond Shao and Steel (1999) and Full (2000). Our goal is to develop the capability in StEPS to obtain an approximate estimate of variance that takes into account the component due to the imputation of missing or invalid values. However, in producing this variance estimate, several considerations are to be balanced: (1) the accuracy of the resulting variance estimates, (2) the ability to generalize the procedure to various types of imputation, (3) the procedure's robustness to the use of mixed imputation, and (4) the ease of implementing the procedure within StEPS. Weighing these constraints, we considered several procedures discussed in the literature, and concentrate on two of them. The first is a simple procedure that inflates the naïve variance estimate by a factor that depends on the amount and type of imputation. Under the second procedure (Kim 2001), we create “pseudo-data,” a second set of responses perturbed enough that commonly applied variance estimation formulae or software will pick up the variability caused by the imputation. Section 2 contains a brief review of the literature and the methods. Some options for imputation under StEPS are described in Section 3. In Sections 4 and 5, respectively, we explore the inflation-factor approach and Kim's method. Finally, in Section 6 we provide results of a simulation study. Many derivations, citations, and other details are left out of this paper, but can be obtained from the author in a separate technical report.